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SUMMARY 

The maximum-likelihood sequence estimation re- 
ception system, where the transmitted signal sequence is 
decided by the maximum likelihood using Viterbi algo- 
rithm, from the modulated signal, such as the phase mod- 
ulated convolution-coded signal, TCM, and CPM, is 
considered to be very interesting because of its excellent 
signal-to-noise ratio (SNR) versus error rate performance. 
Such a system assumes the coherent detection of the 
received signal in order to realize excellent performance. 
The reception system, which is expected to follow the 
carrier with a large offset and a low SNR, has been diffi- 
cult to realize due to the difficulty of satisfying the re- 
quirement This paper proposes the following system. The 
phase-locked loop is provided corresponding to each 
survival path in the Viterbi algorithm used in the estima- 
tion of the maximum-likelihood sequence estimation. The 
phase-locked loop is selected in parallel to the selection 
of the survived path. In other words, the phase synchro- 
nization and the (maximum-likelihood) sequence estima- 
tion are simultaneously achieved. The system is discussed 
in detail, and it is shown that a stable coherent detection 
can be realized for a very low SNR. 

Key words: Satellite communication; mobile com- 
munication; maximum- likelihood sequence estimation; 
convolution^ code; phase-shift keying. 
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1. Introduction 

The maximum-likelihood sequence estimation re- 
ception system is considered in which the phase modu- 
lation of the convolution-coded signal [I, 2], the continu- 
ous phase modulation (CPM) [3-7], the trellis-coded 
modulation (TCM) [8, 9], and the modulation with inter- 
symbol interference [10, 1 1] are used; and the transmitted 
signal sequence ts estimated by the maximum-likelihood 
decision using the Viterbi algorithm, from such a modu- 
lated signal in the wide sense, containing coding. Such a 
system is considered very interesting, because of its ex- 
cellent SNR vs. error-rate performance. The system ex- 
hibits an excellent spectrum efficiency and the error rate 
performance. It is an especially useful modulation system 
in the satellite communication and the mobile communi- 
cation, in which severe requirements are imposed in the 
band and power limitations. The maximum-likelihood 
sequence estimation technique is important in such a 
system. 

The maximum-likelihood sequence estimation using 
Viterbi algorithm is usually applied to the baseband sig- 
nal, which is obtained by the coherent detection of the 
received signal [1,2, 10}. In the satellite communication, 
the demodulator must work for the low SNR due to the 
severe limitation of the transmitter power. The demodu- 
lator must work for a still lower SNR in applications such 
as deep-space exploration where the channel coding with 
a high error-correcting ability, such as the conventional 
code with a long constraint length or the concatenation 
code, is used In the case of the cellular mobile commu- 
nication system, it is required that the receiver should 
work correctly for the severe condition (such as low CIR 
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low C1R and SNR). in order to reduce the number of 
cell iterations and to utilize the spectrum effectively. One 
of the major reasons in the deterioration of the receiver 
performance in the low SNR condition is the generation 
of the unreceivable state due to the cycle slip of the 
regenerated carrier, in addition to the increase of the 
error rate due to the phase jitter of the regenerated carri- 
er. The absolute-coded phase modulation, without the 
differential coding, exhibits a better SNR vs. error rate 
performance, compared to the case using the differential 
coding; it is widely used in the satellite communication 
system where the transmitter power is severely limited. 

In the system without differential coding, the 
known signal is inserted at regular intervals in the trans- 
mitted sequence in order to indicate the absolute phase of 
the transmitted signal. Based on that signal, the phase 
uncertainty is eliminated and the absolute phase is estab- 
lished. When a desynchronizaiion occurs, the demodu- 
lator cannot work correctly until the synchronization is 
re-established at the same absolute phase before the 
desynchronizaUon (error rate « 0.5). Even if the syn- 
chronization is re-established after the desynchronizaiion, 
the demodulator does not work correctly until it receives 
the next known signal if there is produced a cycle slip in 
which the synchronization is established at an absolute 
phase which is different from the phase before the desyn- 
chronization. Thus, after receiving the known signal, the 
phase synchronization must be maintained in the coherent 
detector of the receiver until the next known signal is 
received while suppressing the generation of the cycle 
slip. In other words, the probability for the cycle slip 
generation should be suppressed to a very low value, 
especially when the receiver must operate for a low 
SNR. 

In the satellite communication or the mobile com- 
munication, there exists a large frequency offset or the 
carrier phase variation, due to the inaccuracy of the 
synthesizer frequency, the fading or the Doppler shift 
due to the motion of the satellite. Consequently, the 
coherent detector to be used in those systems must follow 
the large and rapid variation of the carrier phase. 

A typical method to synchronize the regenerated 
carrier to the received signal follows. The coherent 
detection using the regenerated carrier is applied to the 
received signal. The phase error is extracted by eliminat- 
ing the modulation component from the obtained base- 
band signal. Using the signal obtained by eliminating the 
noise component from the error signal by the loop filter, 
the phase of the regenerated carrier is controlled. The 
noise immunity and the follow-up performance to the 



phase variation in such a system are determined by two 
basic functions: one to eliminate the modulation compo- 
nent from the received signal (the phase error detection 
function), the other to eliminate the bandwidth of the 
loop filter or the tank circuit. 

In general, it is considered that the loop filter of 
the coherent detector should be designed to be wide, in 
order to realize a wide pull-in frequency range and to 
follow up more easily to the phase variation (such as the 
fading and Doppler shift). On the other hand, the band- 
width of the loop filter in the coherent detector should be 
sufficiently narrow in order that the detector can operate 
for a low SNR. In other words, it is difficult to satisfy 
these two requirements, i.e., good noise immunity and 
good follow-up to phase variation, by manipulating only 
the parameter of the loop filter. 

As to the other function, i.e. , the elimination of the 
modulation component, there is no problem if the modu- 
lation signal component is known. However, it is un- 
known in the coherent detector that tries to detect the 
transmitted signal. Consequently, in the conventional 
system, the modulation component is estimated by some 
means and the phase error signal is derived based on that 
estimation. Then the performance of the modulation 
component estimation is the decisive factor in determin- 
ing the performance of the phase detector. In such a 
conventional system, the error rate of the transmitted 
signal estimation in the output of the coherent detector is 
very high for the low SNR. This prevents satisfactory 
detection of the phase error and causes deterioration of 
the coherent detection performance. 

Other than the foregoing system, where the co- 
herent detection and the maximum-likelihood estimation 
are separately provided, there are efforts to improve the 
receiver performance by integrating those functions (12- 
15]. Such a system, however, has the following prob- 
lems. The sequence estimator with a large delay is in- 
cluded in the coherent detector which deteriorates the 
loop response and prevents the realization of a satisfacto- 
ry follow-up to the phase variation [12, 13]. The carrier 
phase is handled as the state in the Viterbi algorithm and 
the receiver performance is greatly deteriorated when 
there is a large frequency offset [14, 15]. 

From such a viewpoint, this paper proposes the 
following system where the phase synchronization and 
the (maximum-likelihood) sequence estimation are simul- 
taneously executed. The phase -correction loop is provid- 
ed corresponding to each of the survived paths in the 
Viterbi algorithm to be used in the maximum-likelihood 
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Fig. 1. Transmission system model. 



sequence estimation. Then the phase-correction loop is 
selected simultaneously with the survived path. In this 
system, the modulation component can be estimated with 
a high probability even for a low SN condition, and it is 
expected that the synchronization performance will be 
greatly improved for the low SNR. A special advantage 
is mat a large delay is not included in the phase-correc- 
tion loop; the follow-up performance is satisfactory even 
if there is a large phase variation. 

The proposed system is discussed in detail in the 
following. It is shown by simulation that an excellent 
demodulation performance is obtained when the method 
is applied to the phase -modulated convolutionally coded 
signal, even if the SNR is low and there is a large fre- 
quency offset. Section 2 presents the channel model and 
the problems in the conventional coherent detector. Sec- 
tion 3 outlines the method to integrate the maximum- 
likelihood sequence estimation and the phase detector. 
Section 4 discusses the application of the Viterbi algo- 
rithm. Section 5 presents the result of exaniinauon by 
simulation. 

2. Transmission System Model and Con- 
ventional Coherent Detector 

2. 1. Transmission system model 

The transtnitter/receiver system is modeled as in 
Fig. I. The uTmsmission data sequence {a t : i » -*~co, 
a ( is rM bit data} is encoded (with coding ratio r) and 
modulated (M bit/symbol). It is affected by certain chan- 
nel characteristics as well as by the additive Gaussian 
noise (AGWN). The quadrature demodulation is applied 
using the asynchronous carrier. The signal is sampled by 
the symbol rate (1/7), and the sampled sequence fo: i « 
-oo—oo} is obtained. It is assumed that the sampling 
clock is synchronized in phase to the transmitter clock. 



The task of the proposed system is to estimate the trans- 
mitted data sequence {a t : i a -oo - co} from the sampled 
value sequence {z,-: i = -oo~co}. 



2.2. Received sampled sequence 

It is assumed that the input/output characteristics 
(constraint length v) of the modulator and the encoder , 
as well as the impulse response lengths of the channel 
and the transmitter/receiver filters, are finite. The sam- 
pled sequence is given as follows: 

*-/,<*,—,, .*<>*exp(-;W7))+ n (0 

where f { (c { m _ ~, a t ) is the overall transmission 
characteristics, determined by the coding method, modu- 
lation method, and the impulse responses of the transmit- 
ter/receiver filters. B(i7) is the phase difference between 
the unmodulated carrier and the local oscillator output of 
the receiver, n ' is the complex Gaussian noise affected 
by the receiver filter. When the transmitted signal is the 
phase-modulated convolution^ code, ffa i . , . a f ) 
takes one of the following values. 

exp(~;W2*-'):H==0. 1, ,2*-l 

The conventional modulation/demodulation system 
works as follows. Using the coherent detector, the above 
phase component exp(-; 6(i7)) is compensated and 
modulation-eliminated. Based on the obtained demodulat- 
ed baseband signal 

* 

Sr=z i exvU[8VT)+6<f>)) 
~f,{a..y*i, .rt<)exi>0'<S#) 



84 



MLF : Maximum likelihood function 
RS : Reference sequence 
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Fig. 2. (a) Conventional QPSK coherent demodulator and decoder; and (b) phase error detector 

characteristic of conventional QPSK coherent demodulator. 



the transmitted data sequence {a s : i =» -oo-oo} is esti- 
mated using the maximum-likelihood sequence estima- 
tion. In the forementioned, 6<p is the phase error 
contained in the output of the regenerated carrier 
oscillator. When the SNR is sufficiently high in general, 
there holds 

AbsC^+j/'X^^ (3) 

in the steady state of the coherent detector. In the forego- 
ing, Abs(x) is the absolute value of x. n is the noise 
phase component due to the effect of the noise n'exp{/ 



23. Conventional coherent detection and prob- 
lems 

Figure 2(a) shows conceptually the conventional 
coherent detection system. As is shown in the figure, the 
conventional coherent detector extracts the phase error of 
the regenerated carrier from the demodulated baseband 
signal and the phase of the regenerated carrier is con- 
trolled by the phase error signal. Usually, the phase error 
of the regenerated carrier is extracted by deciding the 
demodulated baseband signal as one of cxpi-jnn/ 
2 1 *' 1 ): r«0, 1, 2, ~, 2 W - 1 and by deteniiining the 
phase difference between the decided value and the 
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demodulated baseband signal. In practice, the transmitted 
signal is coded and the signal cannot always take any 
value of exp(-yii7i/2 u ' '): n = 0, 1, 2, ~, 2 m - 1. 

The phase error is thus obtained by passing the 
received baseband signal Sr through the periodic saw- 
tooth phase response with zero values at n%/2 M " l : n = 
0, 1,2, 2 W - 1 (as is shown in the phase error detec- 
tor of Fig. 2(b) for AS «2), In the past t the 2 A/ -muttipIi- 
er is known as a method for realizing such a response. 
The result obtained by such a response is a litlie 
different, but is essentially the same. 

The foregoing method has no problem in the opera- 
tion when the SNR is high enough to guarantee Eq. (3). 
In other words, when the SNR is sufficiently high, there 
is little decision error in the value decided by the coher- 
ent detector. The phase difference between the decided 
value and the received baseband signal then is the sum 
o<j>+/j"ofthe phase error component and the noise. By 
deleting the noise component from the sum signal using 
a Miter, the phase error component can be extracted with 
a sufficient accuracy. 

When the SNR is low, on the other hand, the 
decided value may contain various errors. When the 
four-phase PSK operates near Ej/rV 0 , the error rate is 
approximately 10 percent. In this case, the detected 
phase error contains the detection error due to the deci- 
sion error by a ratio of 10 percent. In the case of a 
decision error, 8 value entirely different from 6<}> + n is 
inputted from the phase error detector. 

Consider, for example, the case where a decision 
error is produced in the four-phase PSK when the error 
signal exceeds only slightly the decision threshold. If the . 
phase error detection response of Fig. 2(b) is used in 
such a case, the magnitude and the sign of the detected 
phase error signal greatly differs, depending upon wheth- 
er the baseband signal exceeds or does not exceed the 
threshold. In other words, instead of extracting the phase 
error 6$ +n\ H ♦ « ± Abs(o<fr + n ± tc/2) < 
ic/4 may be ourputted. This detection error is a nonlinear 
"noise," which produces a large deviation from the true 
phase error. Thus, in the widely used conventional co- 
herent detector, this nonlinear noise drastically deterio- 
rates the phase synchronization performance when there 
are produced a large number of decision errors in the 
phase error detector. This is the factor that prevents the 
satisfactory operation of the system in low SN condition. 

When the convolution code of constraint length 7 
and rate 1 12 is used in the four-phase PSK. even if there 



is an error of some 10 percent before the decoding, the 
error rate is reduced to approximately 0. 1 percent after 
the decoding. In the communication system that operates 
in practice around this error level, the coherent detector 
must cope with the error rate around 10 percent. In other 
words, there must be a coherent detector that realizes a 
satisfactory operation in the adverse condition as 
described here. This implies that a more severe condition 
is imposed on the coherent detector as the error-correct- 
ing ability is more improved. 

2.4. Coherent detector for tow SNR 

Using the foregoing reasoning, it can be seen that 
one of the factors that prevents the satisfactory operation 
of the coherent detector for the low SNR is the decision 
error in the phase error extraction in the coherent detec- 
tor. One way to remedy the performance deterioration 
due to the decision error may be to utilize positively the 
convolution code multiplied with the transmitted signal 
and the maximum-likelihood sequence estimation to 
reduce the decision error. 

In general, it is inevitable that a considerable delay 
is produced in the system to decode or demodulate the 
convolution^ code or the intersymbol interference. When 
the maximum-likelihood sequence for the convolution^ 
code is estimated by the Viterbi algorithm, a decoding 
delay of several times longer length compared to the 
constraint length is considered as necessary, especially 
for the low SN condition. 

If such a tnaximum-likelihood sequence estimator 
is directly applied to the decision unit of the coherent 
detector, a very large delay is included in the phase- 
locked loop. A large delay in the loop affects the stability 
of the loop and deteriorates the synchronization perfor- 
mance. In other words, a large improvement of the 
performance cannot be expected if the maximum-likeli- 
hood sequence estimator is directly included in the coher- 
ent detector. In the following sections, a method is pro- 
posed and investigated in which the maximum-likelihood 
sequence estimation with the low error rate is applied to 
the decision-making in the coherent detector without 
deteriorating the synchronization performance. 

3. Maximum-Likelihood Sequence Esti- 
mator with Phase-Tracking Function 

In order to help understand the maximum-likeli- 
hood sequence estimation method using the Viterbi algo- 
rithm discussed in the next section, assumptions are 
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presented in different stages, together with some accom- 
panying discussions. In the following, section 3.1 dis- 
cusses the sequence estimation assuming a complete 
phase synchronization. Section 3.2 describes the phase 
(correction) synchronizer assuming that the transmitted 
sequence is known. Section 3.3 discusses the sequence 
selection , after applying the phase correction to each 
candidate sequence that has the possibility of having been 
transmitted. 

The first step is to consider the estimation of the 
transmitted sequence a{. i = 1 , p from the sampled 
sequence z,: j = 1, ~, p. The maximum-likelihood esti- 
mation is the following procedure. When a signal is 
received, the transmitted data sequence with the highest 
probability (likelihood) is sought, among all data 
sequences (2 P in this case) that have the possibility of 
having been transmitted. The data sequence with the 
highest probability is decided as the transmitted 
sequence. In other words, in the maximum-likelihood 
decision, the likelihood is calculated for each of the data 
sequences that has the possibility of being transmitted 
and the transmitted data sequence is selected as the se- 
quence with the highest probability. 

The sampled sequence ^: / « 1, p is the output 
from the quadrature detector using the local oscillator 
that is not phase synchronized. It contains the phase 
difference 8(i7) between the carrier and the local oscilla- 
tor output of -the receiver and cannot directly be used in 
the estimation of the transmitted sequence a,: r = 1, -•, 
p. To estimate the transmitted sequence a { \ i - I, », p, 
it must be the case that either the phase difference 6(/7) 
is known or is estimated by some means. 



3.1. Sequence estimation assuming a complete 
synchronization 

As the first step, consider the case where a com- 
plete synchronization is established between the phase of 
the carrier and the phase of the local oscillator of the 
receiver. 

0(iT)«O W 

Then the likelihood of each of the h - 2 P data sequences 
that may have been transmitted can be determined as 
follows. Assuming that each sequence is transmitted, the 
distance between the actual received signal and the signal 
point (reference signal) that will be obtained if the ideal 
signal is received, is calculated. 



Then the likelihood can be determined uniquely 
based on the sum of the square distances: 

p 

■ 

The likelihood is high if the foregoing square-sum dis- 
tance is smaller. Here, rf 1 is each of the h « 2? signal 
sequences that have the possibility of having been trans- 
mitted. There are obtained correspondingly 2 P likeli- 
hoods. Among those 2 P likelihoods, the maximum is 
determined and the sequence corresponding to the maxi- 
mum is ourputted as the sequence with the maximum 
likelihood. 

The calculation of the likelihood shown in Eq. (5) 
based on the distance and the sequence estimation based 
on the likelihood correspond to the maximum likelihood 
sequence estimation in the state where the conventional 
coherent detector is operating correctly. As was dis- 
cussed, the synchronization performance of the conven- 
tional system in Fig. 2(a) deteriorates when there is a 
decision error in the decision unit in the phase error 
detector, which prevents the calculation of the likelihood 
shown in Eq. (5) as well as the sequence estimation 
based on the result 



3.2. Phase synchronizer assuming known 
transmitted sequence 

As a next step* consider the case in which the 
complete phase synchronization is not established be- 
tween the local oscillator of the receiver and the carrier. 
Assume that the transmitted signal is known in the re- 
ceiver by some means. In this case, the known transmit- 
ted signal sequence can be used instead of the output of 
the decision unit in Fig. 2(a). This corresponds to the 
case in which no error is included in the decision output. 

Then, corresponding to the transmitted data se- 
quence, the sequence (6(f7): / = 1, p) of the phase 
difference between the local oscillator output and the 
carrier can be estimated. The estimated value for 8(/7) 
is the output of the loop filter when the loop of the co- 
herent detector of Fig. 2(a) is cut at the output of the 
loop filter. Estimation for our purposes means that the 
effect of the noise is considered. The phase error se- 
quence can be estimated if the SNR is infinite. When the 
SNR is infinity, the phase sequence is represented as 
follows: 

*bT)-ArgU)-Ai«(fv) (6) 
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The synchronizer operates in order to compensate 
the foregoing phase difference. This corresponds to the 
case in which the loop of the coherent detector is closed. 
Then the output phase of VCO tends to compensate 
0(i7). In practice, however, the transmitted data 
sequence is unknown, whichi is the original problem. 
This point is considered in the next section. 



the likelihood for the h-th sequence is determined from 
the phase-corrected input signal sampled sequence de- 
rived based on the A-th sequence, together with the refer- 
ence signal for the h-th sequence. The likelihood is 
calculated based on the square-sum of the distance 



3.3. Sequence selection after phase correction 
of each sequence candidate with possibility 
of transmission 

Assume that a sampled sequence z(. i « 1, ■-, p is 
obtained. There 2 P data sequences that have the possibili- 
ty of having been transmitted. For the given sampled 
sequence, an arbitrary one of the 2 P data sequences that 
have the possibility of having been transmitted is selected 
and the phase synchronizer is operated. This is the same 
as operating the coherent detector, using an assumed 
sequence r* of 2? sequences instead of the decision 
output (in PED) of Fig. 2(a). In this case, the coherent 
detector operates to compensate the phase error sequence 
(e A (/7): x » 1, p) 

0*(*T) = ArgU)-Arg(r*) (7) 

based on the assumed sequence. The output of this coher- 
ent detector is the phase corrected input signal sampled 
sequence 

z?=z, *exp(-f;#) (8) 

* 

In the foregoing, ♦ * is the phase generated inside 
of the coherent detector so that the coherent detector 
compensates the phase error sequence &(iT). It is ob- 
tained by integrating the output of the loop filter in the 
coherent detector. In other words, is the result of 
deleting the nonstationary component, such as noise, 
from «*(i7). 

Such e phase-corrected input signal sampled se- 
quence can be determined for each of the h « 2 P assumed 
sequences. In other words, h-2 p phase-corrected input 
signal sampled sequences can be determined. One of 
those h = 2 p assumed sequences is the true transmitted 
sequence. In other words, one of the 2 P phase-corrected 
input signal sampled sequences is the input signal sam- 
pled sequence, which is the result of coherent detection/ 
phase correction based on the true transmitted sequence. 

The likelihood can be calculated for each phase- 
corrected input signal sampled sequence. In other words, 



The likelihood is defined as large when L n * is small 

If the "assumed transmitted data sequence h' is the 
same as the actually transmitted data sequence, the 
phase-corrected input signal sampled sequence z* is the 
same as the sampled sequence when the phase synchroni- 
zation is established between the local oscillator of the 
receiver and the carrier. In that case, the square-sum of 
the distance in Eq. (9) is the same as that of the case 
where the transmitted sequence is assumed as known, as 
in section 3.2. 

When the likelihood is calculated based on Eq. (9), 
for all possible transmitted data sequences, one can 
estimate for the received signal which of the possible 
transmission sequences has the highest probability of 
having been transmitted. At least, if SNR * °o, the maxi- 
mum likelihood is obtained for the actually transmitted 
sequence. Also, in the actual case where SMR * oo, the 
likelihood for the data sequence calculated from the 
phase-corrected sampled sequence based on the correctly 
assumed transmitted data sequence is the maximum with 
a high probability compared to the case where other 
sequences are assumed. 

By such a scheme, the estimation of the transmitted 
data sequence and the estimation of the phase error 
sequence, i.e., the maximum likelihood estimation and 
the phase synchronization, can be executed simultaneous- 
ly. Figure 3 shows the proposed system where the phase 
synchronization and the sequence estimation are simulta- 
neously executed. The principle is realized by providing 
2 P phase-correction loops. In Fig. 3, the phase-error 
detector (PED) in the individual phase-correction loop 
differs slightly from the PED shown in Fig. 2(a). The 
difference is that PED in Fig. 2(a) contains internally the 
decision unit while thai of Fig. 3 does not contain a 
decision unit and the reference signal (RS) is used instead 
of the decision output. 

In such a system, the phase synchronization is 
maintained for at least one of a very large number of 
assumed transmitted data sequences, which is the result 
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of the phase correction of the actually received signal, 
assuming the maximum-likelihood transmitted data. 

4. Phase-Tracking Viterbi Demodulation 

Algorithm 

If the foregoing series of operations are directly 
implemented, a tremendous amount of transmission data 
sequences and the pbjase-coirection loops must be pre- 
pared, which will not be realizable. To realize the same 
operation with less hardware, the Viterbi algorithm is 
applied. Viterbi algorithm is the algorithm that retains 
the specified number of survived paths and decides the 
transmission sequence with the maximum likelihood by 
gradually updating the path metric for each path. 

To combine the phase-tracking function with the 
conventional Viterbi algorithm, the following two mecha- 
nisms are newly add e d. 

0 The phase-correction loop function to corre- 
spond to each survived path. 

© ACS operation is applied based on the calculat- 
ed branch metric. The path as welt as (the internal state 
of) the phase-correction loop are selected so that the 
reference signal for the new state and (the internal state 
of) the phase-correction loop are generated. 



Let there be at least 2* " 1 states of the transmis- 
sion code trellis (v is the constraint length). The individ- 
ual phase-correction loop corresponds to each of the 
survived paths of the state. The phase-correction loop is 
considered as the secondary loop. Consequently, the 
internal state of the phase-correction loop at time i can be 
completely represented by the phase correction d)*: h - 
1, V ' 1 and the frequency offset ff\ h « I. V " \ 
which is the first-order derivative of the phase correc- 
tion. In each phase-correction loop, the following phase 
rotation is applied to each input sampled value based on 
the phase correction 

*exi>( rv#) (10) 

Thus, different phase corrections are applied to the 
obtained states. Based on the input sampled sequence 
after the phase correction, the branch metric Bmu!\ is 
calculated as 

Bw^r^Uf.i-r,^! 1 (ID 

In the forementioned. 0m?.m is the branch metric 
corresponding to the branch from state k to state m in the 
shift from the i-th sample to the (/ ♦ l>th sample. 
rt.7"\ is the reference signal corresponding to the 
transition from state k to state m, in the shift from the /- 
th sample to the (i * l)-th sample. It is calculated in the 
receiver for the ideal case. 
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Fig. 4. (a) Basic calculation of conventional Viterbt algorithm, and (fa) 
basic calculation of phase-tracking Viterbi demodulator. 



In parallel to the calculation of the branch metric, 
the phase error data d<f> h n'Fis determined as follows: 

« Ars<*?„) - Ar«( ri/. 1 !) (12) 
where A+U"$\k. 



Based oa the path metric for each state and the 
branch metric obtained here, the ACS operation is ap- 
plied to determine the new path. The path metric of each 
state is updated as well as the phase correction. The 
phase correction * is controlled and memorized as 
follows, based on the phase error d<f>UT determined for 
the transition from state k to state m. 
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(13a) 
(13b) 



Since the phase-tracking loop is considered as the sec- 
ondary loop, the frequency data/.* is also memorized as 
the internal state of the loop, in addition to the phase- 
correction data, a and P correspond to the parameters of 
the loop filter. They have the same physical significance 
as the parameters of the loop filter in the conventional 
coherent detector shown in Fig. 2(a). 

Figures 4(a) and (b) show conceptually the conven- 
tional Viterfai algorithm as well as the stepwise ACS 
operation processing of the proposed Viterbi demodula- 
tion algorithm with the phase-tracking function. As is 
seen from the figure, the path is determined by selecting 
the larger attached path metrk The branch metric re- 
quired in this process is calculated based on the sampled 
value i\ which is phase-corrected by the carrier data in 
the phase-correction loop corresponding to the survived 
path h arriving at the start state of that branch metric as 
well as the reference signal, which is newly calculated 
using the reference signal corresponding to the survived 
path arriving at the start state of that branch. 

The branch metric is calculated for more than one 
(two in general, as in the case of the convolution*! code) 
branches arriving at a state. The result is added to the 
path metric of each start state of those multiple branches. 
The results are compared, and one of the branches is 
selected as the survived path. As is described before, the 
phase correction and the reference signal are determined 
corresponding to the selected path. Then, using the trans- 
mitted data corresponding to the selected branch, the 
phase error signal is calculated from the sampled value 
and the phase correction is applied to the start state of 
the selected branch. The result is used as the new carrier 
data for the survived path. 

By applying the Viterbi algorithm, the data 
sequences which obviously have smaller possibilities of 
having been transmitted compared to other sequences, 
are gradually discarded. At the same time, the synchroni- 
zation loops coaesponding to those sequences are also 
discarded. By this scheme, the same performance as the 
phase tracking and sequence estimation system shown in 
Fig. 3 is obtained. 



5. Simulation 

The validity of the proposed method is verified by 
a computer simulation. It is shown in the following that 



the proposed method where the rnaximum-likelih 
estimator containing the phase-tracking function can 
realize a stable demodulation in the very low SN condi- 
tion. The overall error rate performance can be 
improved, compared to the conventional system outside 
of the receiver, where the coherent detector and the 
maximum-likelihood sequence estimator are serially 
placed and operate independently. 



5.1. Simulation condition 

The case is considered where the transmitted signal 
sequence is convolutionally encoded and is transmitted by 
the absolute phase four-phase PSK. The performance of 
the proposed method is examined by a computer simula- 
tion. The conventional code of constraint length 5 and 
rate 1/2 is used. The 2-bit output signal produced for 
each input bit to the encoder are mapped to the in-phase 
and quadrature channels of the same timing in the four- 
phase PSK. As the transmitter/receiver filters, the root- 
roll-off filter (roll-off factor 30 percent) is assumed, and 
the Nyquist response is assumed for the overall response 
of the transmission channel. The SNR is always repre- 
sented by the carrier-to-notse ratio (CNR) after passing 
through the receiver filter. 

The known signal is inserted once in each 1000 
symbols of the modulated signal (1000 bits of the signal 
before encoding), and the phase uncertainty is assumed 
to be eluninated using the known signal. Consequently, 
1000 symbols at the maximum produce the undecodable 
burst error when there is produced a cycle slip or the 
normal phase tracking is made impossible. In the follow- 
ing, the above 1000 symbols plus the known signal is 
defined as a slot. 

In the phase-tracking Viterbi algorithm proposed in 
this paper, the phase correction corresponding to the 
survival path is selected. Consequently, the phase uncer- 
tainty, which has been a problem in the conventional 
absolute phase four-phase PSK and in other similar sys- 
tems, is eliminated. In other words, the known signal to 
eliminate the phase uncertainty is umiecessary in princi- 
ple. In the conventional system, which is to be compared 
to the proposed system, the known signal must be insert- 
ed in order to eliminate the phase uncertainty. In addi- 
tion, the synchronization performance is in general im- 
proved by inserting the known signal. As a result, it is 
assumed that the known signal is inserted into the modu- 
lated signal so that the conventional system and the pro- 
nrtcM *vstem can be compared under the same condition. 
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Fig. 5. Bit-error rate for phase-tracking Viterbi 

demodulator. 



It is assumed that the known signal is sufficiently long 
and the phase uncertainty can be completely diminated. 

Consider the situations where the number of error 
bits in a slot exceeds 200 and 50 after the convolution 
decoding. They are called the 20-percent error slot and 
the 5-percent error slot* respectively. In the very low 
SNR condition, where the desynchronization and the 
cycle slip occur frequently and continuously, it is diffi- 
cult to distinguish between the desynchronization and the 
cycle slip. Then it is practically impossible to measure 
the number of times for the cycle slip. Consequently, the 
generations of the 20-percent and 5-percent error slots 
are used to represent the generation of the cycle slip. 

The generation of the 20-percent error slot indi- 
cates that a cycle slip or a desynchronization is produced 
soon after receiving the known signal. The generation of 
the 5-percent error slot indicates that a cycle slip is 
produced at a certain elapse of time after receiving the 
known signal. In other words, when a large number of 
20-percent error slots are generated, it cannot be expect- 
ed that the overall error rate is greatly improved even if 
the slot length is decreased. When a large number of 5- 
percent error slots are generated but there 8re fewer than 



10" 



S 

CO 

I 10* 
ta 



10* 3 



C/N 

Fig. 6. Bit-error rate for phase-tracking Viterbi 

demodulator. 



20-percent error slots, it can be expected that the overall 
error rate is improved by carefully determining the slot 
length. 

5.2. Result of simulation (error rate perfor- 
mance) 

Figures 5 md6 show the error rate performances 
of the proposed Viterbi demodulator with the phase syn- 
chronization (unction (PTVD) and the conventional de- 
modulator (CD) composed of a serial connection of the 
coherent detector and the maximum-likelihood sequence 
estimator, respectively. 

In the measurement of the error rate performance, 
it is assumed that the constant frequency offset is pro- 
duced in each slot (immediately after receiving the 
known signal). In other words, the phase varies in a 
ramp form. This corresponds to the situation in the 
satellite conununication and deep-space exploration where 
the satellite or the explorer modifies the posture or orbit 
while executing transmission/reception* It also corre- 
sponds to the situation in mobile satellite conununication 
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in which the mobile station changes the direction of 
motion. In such cases, the receiver must follow the 
Doppler-shift and other changes produced by the varia- 
tion of the relative velocity, with a sufficient tolerance. 
The figures show the performance for various frequency 
offsets. The frequency offsets in Figs. 5 and 6 corre- 
spond to 0.5 percent and 0. 1 percent of the transmission 
rate, respectively. 

a and P, which are the coefficients of the loop 
filter, are set as the same values in the two systems. The 
relation (a/2) 1 = j* is maintained between a and This 
relation is the condition for the critical damping in the 
secondary tracking detector loop shown in Fig. 2(a). 

It is seen from the figure that the conventional 
system can hardly maintain the synchronization when 
CNR goes below 3 dB, resulting in the error rate of 0.5 
while the proposed system maintains a sufficiently low 
error rate. In other words, it is seen from the figures that 
the proposed system exhibits a much lower error rate 
than the conventional system, especially when SNR is 
low In Fig- 5, the proposed system exhibits the best 
performance for a « 0.0S. In other words, die optimal 



performance for the frequency variation of some 0.5 
percent of the transmission rate, from the viewpoints of 
handling the phase variation and the noise immunity, is 
given by this parameter. This performance cannot be 
achieved in the conventional system; however, the coeffi- 
cient of the loop filter may be adjusted. When the fre- 
quency offset is 0. 1 percent of the transmission rate, on 
the other hand, the best perfonnance is obtained in either 
system for a = 0.04. Especially, in the proposed system, 
the overall error rate which is almost the same as that of 
the ideal synchronization performance (hard-wired carri- 
er) is obtained. 

Figures 7 and B show the generation probabilities 
of the 20-percent and the 5-percent error slots, respec- 
tively, as functions of CNR. It is again seen from the 
figures that the proposed system has a better noise immu- 
nity and handles the frequency offset better. Especially, 
it is seen that the generation probabilities for the 20- 
percent error slot and the 5-percent error slot differ 
greatly. This indicates that the overall error rate will be 
further improved by reducing the slot length or other 
elaborations. In all cases of Figs. 5 to 8, the error rate 
and the probability of cycle slip rapidly decrease when 
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the CNR readies 6 to 8 dB. In this simulation, the trans* 
mission is tried for more than 1 million symbols in either 
case. For the high SNR (>8 dB), no error or cycle slip 
was found. In other words, by applying the proposed 
method, the SNR, for which the receiver is to operate, 
can be set much lower. 



S3. Result of simulation (synchronization per- 
formance) 

In order to verify that the satisfactory pull-in of the 
synchronization can be realized for the low SNR, the 
initial synchronization performance of the proposed 
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system is examined. Figures 9 and 10 show the phase 
synchronization process in the proposed system. Figure 
9 is the case where the stepwise phase variation is im- 
posed, and Fig. 10 is the case where the stepwise fre- 
quency variation is imposed. Figure 9 shows which 
memorized state has the highest likelihood, among the 
memory phases for the states as a function of the time. 
Figure 10 shows the value memorized in the state corre- 
sponding to the maximum likelihood, among the loop- 
filter registers for the states (frequency memory). 

When the state is selected incorrectly, there is a 
possibility that the phase and the frequency offset may be 
outputted, which differs from those used in the demodu- 
lation of the actual output demodulated signal. This can 
be considered as the case where a large phase or fre- 
quency variation occurs, immediately after the stepwise 
variation of the phase or frequency. It is seen from the 
figures that the control progresses toward the goal value, 
and the synchronization is gradually established. 

One hundred trials are made for the forementioned 
synchronization puIJ-in process, and the same pull-in 
performance is obtained for all trials. Thus, it is verified 
from those figures that a satisfactory phase pull-in per- 
formance is obtained even for the low SN condition. 



6. Conclusions 

This paper proposed and examined the Viterbi 
demodulation system with the phase-tracking function for 
the phase modulated convolution code. It is shown that 
the proposed system operates well for the low SN and a 
large carrier phase variation, where it has been difficult 
or practically impossible in the conventional system to 
maintain the phase synchronization. Especially, it is 
shown that the proposed system is better than the conven- 
tional system in the very adverse condition with the CNR 
below 3 dB. The basic performances of the proposed 
system are demonstrated. 

The following problems are left for the future 
study. The puli-in process of the system proposed in this 
paper should be analyzed in more detail. The perfor- 
mance, when the proposed method is applied to various 
modulation/coding system such as CPM and TCM should 
be evaluated and discussed. The method should be com- 
pared to the case where the error-correcting code is 
applied to the coherent detection by an approach different 
from the method in this paper [18]. 
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